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Q\ Abstract 

Compressive sensing (CS) allows for acquisition of sparse signals at 
sampling rates significantly lower than the Nyquist rate required for ban- 
l_H dlimited signals. Recovery guarantees for CS are generally derived based 

^ on the assumption that measurement projections are selected indepen- 

O dently at random. However, for many practical signal acquisition appli- 

cations, this assumption is violated as the projections must be taken in 
I groups. In this paper, we consider such applications and derive require- 

ments on the number of measurements needed for successful recovery of 
QO signals when groups of dependent projections are taken at random. We 

t—{ find a penalty factor on the number of required measurements with re- 

r- H spcct to the standard CS scheme that employs conventional independent 

measurement selection and verify the predicted penalty through simula- 
tions. 

o 

^ 1 Introduction 

Modern receivers and sensors need to process an enormous amount of band- 
width to satisfy continuously growing demands on communication and sensing 
5h systems. Due to the complexity and high power consumption of hardware at 

large bandwidths, a number of innovative approaches for signal acquisition have 
recently emerged, including a class based on compressive sensing (CS). In CS 
approaches, the full signal bandwidth is not converted, hence avoiding the costly 
hardware; rather, prior knowledge of a concise signal model allows the recovery 
to focus only on signal aspects relevant to feature extraction. In particular, if 
there exists a basis in which a signal of interest can be represented sparsely 
(i.e., it can be fully characterized with a small number of coefficients), then it 
is possible to obtain all information needed for successful reconstruction of the 
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Figure 1: Left: Independently random 2-D sampling. Right: Radial acquisition 
trajectories used for MRI, which group measurement selections into slices of the 2-D 
Fourier domain. 

signal from a relatively small number of randomized incoherent measurements 
[f . This number is often much smaller than the number of samples implied by 
the Nyquist sampling rate for representation of all bandlimited signals. 

Most CS contributions assume independent randomness in the measurement 
projections that is exploited to derive bounds on the number of projections 
needed for successful recovery. However, for many practical signal acquisition 
applications, this assumption is violated as the projection measurements must 
be selected in groups. As an example, consider Magnetic Resonance Imag- 
ing (MRI), where the measurements in the 2-D Fourier space cannot be taken 
at random but need to follow sampling trajectories that satisfy hardware and 
physiological constraints: the radial acquisition trajectories of MRI shown in 
Fig. [l] are known to be especially suitable for high-contrast objects. Using such 
sampling trajectories clearly introduces structure into the measurement process 
and hence violates a key assumption underlying the standard analysis of CS 
schemes. 

In this work, we derive bounds on the number of measurements needed for 
successful recovery of signals when the random projection measurements are 
structured into predefined groups. We introduce a metric that upper bounds 
the multiplicative penalty on the number of required measurements introduced 
by grouping with respect to conventional CS acquisition employing indepen- 
dently random measurement selection; such a metric is dependent on the sparse 
signal support and might be useful in the design of many practical signal acqui- 
sition systems with grouped measurement structures. While this metric cannot 
currently be evaluated in a closed form, we employ a computationally feasible 
method that provides lower and upper bounds on its value. We also verify via 
simulations the penalty predicted by the proposed metric. 
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2 Background 



2.1 Compressive Sensing 

Consider the acquisition of an N x 1 signal vector x. Assume that x is known to 
be sparse in some basis; that is, we say the signal x is if-sparse for some integer 
K if x has a representation c = U H x having only K non-zero entries in some 
known orthonormal basis U, although the value and location of those non-zero 
entries may be unknown. In the CS framework, we acquire the M x 1 output 
y = &x, for some M <C N, where <& is the measurement matrix. According to 
CS theory, given certain constraints on $ and M, x can be reconstructed from 
y with high probability. 

2.2 Incoherent Measurements 

Given an orthonormal measurement basis V, a K sparse signal x = Uc, sparse 
in some known orthonormal basis U can be reconstructed successfully from a set 
of M independently drawn random samples ft C {1, ... , N} of y = V H Uc with 
probability not lower than 1 — 5, for any S > 0, as long as the number of samples 
is large enough. Define A = V H U and denote by Aq the matrix built from the 
M rows of A corresponding to the index set f2. Define the coherence n(A) of the 
matrix A as fi(A) = maxtj \A(i,j)\, which has range (J,(A) £ 1] PQ- A pair 
of bases V and U for which the minimal value of n(A) is achieved is referred to 
as a perfectly incoherent pair of bases. 

When the elements of f2 are drawn independently at random, it can be shown 
that the number M of measurements required for successful recovery of sparse 
x depends on the coherence of the matrix A. 

Theorem 1. JI]/ Let A be an N x N orthogonal matrix (A H A = I) with coher- 
ence (i(A). Fix an arbitrary subset T of the signal domain. Choose a subset fl 
of the measurement domain of size \Q\ — M and a sign sequence z on T , both 
uniformly at random over all possible choices. Suppose that 

M > Const- Nfi 2 (A)\T\log(N/8). (1) 

Then with probability exceeding 1 — 5, every signal c supported on T with signs 
matching z can be recovered from y = Aqc q by solving the linear program 

min||c||i s.t. A n c — A n c a . (2) 

c 

Theorem [l] shows that the number of measurements required for successful re- 
covery of a sparse signal scales linearly with the signal's sparsity, but only log- 
arithmically with its length, as long as V and U are perfectly incoherent. 

2.3 Grouped Incoherent Measurements 

In certain applications, the assumptions of Theorem [l] are violated as mea- 
surements must be taken in groups instead of independently at random. More 
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specifically, divide the set of N rows of the measurement basis V into N/ G dis- 
joint groups Qi, i = 1, . . . , N/G, of size G each. Note that it will still be possible 
to take a set of measurements f2 for a signal, following Theorem [T] by select- 
ing M/G groups out of the N/G groups available, independently at random^] 
We say that such a process provides a grouped incoherent measurement scheme. 
Grouped incoherent measurement schemes can be seen as a generalization of the 
standard incoherent measurement scheme used in Theorem [I] by setting G = 1. 



2.4 Example Applications 

Interference- Robust Compressive Wideband Receiver. One important exam- 
ple application for a grouped incoherent measurement scheme is an interference- 
robust compressive wideband receiver]^] If a large communication bandwidth is 
employed, interference is nearly always present. More importantly, it is common 
for the signal of interest to be buried in an interferer that is orders of magni- 
tude stronger. This might force the receiver's RF front end into the nonlinear 
range and cause intermodulation distortion that makes the interference cancel- 
lation methods based on interference null space projection [2] ineffective. As 
an alternative, we may opt to perform sampling only at times in which the RF 
front end is not saturated and exhibits linear behavior, e.g., at times when the 
interferer's value is small. A typical interferer is modulated; therefore, while its 
first few zero-crossings can be considered as random, the remaining set of sub- 
sequent zero-crossings are dictated by the frequency of the interferer's carrier. 
Therefore, a sampling approach that aims to operate within the linear region of 
the RF front end results in a grouped incoherent measurement scheme, in effect 
providing an interference-robust compressive wideband receiver. 

Medical Imaging: There are a multitude of medical imaging applications that 
rely on tomography principles, where CS can be applied to reduce the number 
of measurements required for accurate image recovery [3J; common examples 
include MRI and computed axial tomography (CAT). In tomographic imaging, 
the 2-D image measurements obtained via a tomographic scan correspond to 
samples of the Radon transform of the image. These samples can be grouped 
by orientation and processed in groups via the discrete Fourier transform. Ac- 
cording to the projection slice theorem, the output of this transformation pro- 
vides samples of the image's 2-D discrete Fourier transform along a line running 
through the origin (cf. Fig. [I]) . Thus, the measurements obtained correspond to 
a grouped measurement in the 2-D Fourier transform domain of the image, and 
groups can be selected independently by selecting tomographic scan orientations 
independently. 

Multi- dimensional signals and signal ensembles: For signals spanning many 
physical dimensions, such as space, time, spectrum, etc., it is often difficult to 
design CS acquisition devices that can calculate random projections involving 

x We assume that G divides M and N for simplicity. 

2 We thank Robert Jackson for suggesting this architecture for addressing the interference 
problem in wideband receivers. 
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all signal samples. Instead, it is commonly easier to modify the CS acquisi- 
tion process so that it is applied separately to each piece of a partition of the 
multidimensional signal. Examples include hyperspectral imaging and video 
acquisition, sensor networks, and synthetic aperture radar |U[S]. Consider in 
particular the case where the choice of measurements used for each partition 
comes from a single orthonormal basis and is shared among partitions, intro- 
ducing structure in the measurements. For example, a compressive video camera 
may use the same incoherent projections on each frames in the video sequence. 
The resulting global measurement basis is downsampled in a group-structured 
fashion. The grouped incoherent measurement framework can be applied when 
a single orthonormal basis is used for compression of the entire multidimensional 
signal [1]. 



3 Performance Analysis for Grouped Incoherent 
Measurements 

3.1 Performance Metric 



The grouped incoherent measurement scheme introduced in Section 2.3 violates 
the assumptions of Theorem [T] and causes an increase of the number of measure- 
ments needed for successful recovery of sparse signals. Such a penalty factor 
depends on the structure of the groups Q = {Qi, . . . , Gn/g}> 011 the product of 
the measurement and transformation basis A — V H U, and on the set T defining 
the sparse signal support. We define a penalty factor 

7 (A,T,g)= max ||A^|| , (3) 

where ||M|| p ^. g = max/ ||M/|| g /||/|| p denotes the p —> q operator norm of the 
matrix M, M denotes the matrix M after row normalization, and Ag t x is the 
submatrix of A that preserves the G rows corresponding to the group Qi and the 
\T\ columns corresponding to the sparsity set T . Given the set T defining the 
sparse support, the penalty factor j(A, T, Q) is a measure of similarity among 
the rows of Aq.t for each i. For example, if the rows of Aq.t are equal for some 
i, we will have j(A, T, Q) — G; in contrast, if all rows of Ag t T are mutually 
orthogonal for each i, then we will have j(A, T, Q) = yG. 



3.2 Recovery Guarantees 

We now provide requirements on the number of measurements needed for suc- 
cessful recovery of the sparse signal x when the subset f2 of the measurement 
domain is built in a structured way. The following theorem is proven in the 
appendix. 

Theorem 2. Let A be an N x N orthogonal matrix (A H A — I) with coherence 
fJ.(A). Fix an arbitrary subset T of the signal domain. Choose a subset Q of 
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the measurement domain of size \Q\ = M as the union of M/G groups from 
Q = {Gi, ■ ■ ■ ,Gm/g} an d a s W n sequence z on T, both uniformly at random 
over all possible choices. Suppose that 

M > j(A, T, Q) ■ Const • fi 3 (A)N 3 ^\T\ \og{N/5). (4) 

Then with probability exceeding 1 — 5, every signal c supported on T with signs 
matching z can be recovered from y = A^c by solving the linear program 
for any 8 > 0. 

The theorem shows that for perfectly incoherent measurement and sparsity 
bases, j(A,T, G) provides a multiplicative penalty on the number of measure- 
ments necessary for successful signal recovery due to the grouped structure of 
the incoherent measurement selection. Note that for a group size G = 1 and for 
perfectly incoherent pair of bases V and U our result coincides with Theorem [T] 
as it is equivalent to drawing elements of ft uniformly at random. Additionally, 
when the rows of some Aq.t are identical, the additional measurements bun- 
dled within the group provide no additional information, and we must scale the 
number of measurements by G in the worst case. This observation agrees with 
the penalty value 7 (A, T,G) = G for such a matrix At and group structure Q '. 

3.3 Calculation of the Performance Metric 

For a fixed sparsity set T, we can obtain lower and upper bounds on the value 
of j(A, T, G) by leveraging the Pietsch Factorization theorem |BJ. 

Theorem 3. Each matrix B can be factored as B = FD where D is a nonneg- 
ative, diagonal matrix with trace(D 2 ) = 1 and ||S|| tx) _ > 2 < II-PII2 < -Kp||-B|| 00-1.2 7 
where K p is a constant equal to w 1.25 for the real field and w 1.13 
for the complex field. 

Since ||M||2->-i = \\M H ||oo->-2) thanks to the duality of the operator norms, 
we can find bounds on 7 by performing Pietsch factorization of the matrices 
(Ag^) 11 = FiDi, for i = 1, . . . , N/G, where Di is a nonnegative diagonal matrix 
with trace(Z?f ) = 1. The value of ^(A, T, G) can then be bounded by 

— |— max J J Fi J J 2 <j{A,T,G) < max||F<|| 2l (5) 

J\p i i 

The Pietsch factorization of matrix B can be performed by solving a semidcf- 
inite program [B]. As will be pointed out in the simulations in the next section, 
the looseness of the bound ^ can limit the utility of 7. 

4 Simulations 

In this section, we present simulation results for CS recovery from grouped 
incoherent measurements for signals with sparse Fourier domain representations. 
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Consider a discrete signal s of length N = 1100 and sparsity \T\ = 5%-N, sparse 
in the frequency domain, generated as a product of an orthonormal Fourier basis 
of size N x N and a sparse coefficient vector c with values of non-zero entries 
distributed uniformly: ~ U(—l, 1). Consider two different configurations for the 
grouped incoherent measurements: 

Q 1 : 100 groups of size 11 were constructed such that the first sample of each 
of the groups was chosen out of the first 100 samples of s: {s[l], . . . , s[n]}, and 
the remaining 10 samples for each group were shifted with respect to the first 
sample by multiples of 100. More specifically, Q\ = {i, i + 100, i + 200, ... ,i + 
1000}. This configuration appears in the interference-robust compressive wide- 
band receiver application. The first sample corresponds to a random zero- 
crossing of a modulated interferer. Additional samples correspond to subsequent 
zero-crossings of the interferer's carrier. 

Q 2 : 100 groups of size 11 were constructed such that each group contained 11 
consecutive, adjacent samples. More specifically, Qf — {s[i+(i—l)-ll] : s[i -11]}. 
Such configuration assumes that the samples are taken in sequential bursts. 

Figure [2] shows the relation between the penalty factor j(A, T, Q) from Q 
and the ratio between the number M of samples required for successful re- 
covery for the two described group structures and the number of samples Mq 
required for successful recovery for random sampling. The values shown are 
the minimal number of measurements needed to obtain normalized recovery er- 
ror NRE — \\s — s||/||s|| < 0.001 for 99 out of 100 draws of the measurement 
groups (uniformly at random) and the values of the Fourier coefficients (from 
U[— 1, 1]). Each point of the scatter plots corresponds to a fixed signal support. 
We consider three different classes of signal supports: for the first two types, 
the positions of the non-zero Fourier coefficients are chosen uniformly at ran- 
dom within a sub-band built out of two and four 5%-wide channels, respectively, 
positioned uniformly at random within the entire frequency band; we then com- 
pare their performance against the baseline of signals with unrestricted sparse 
supports. Figure [2] shows that for the first two classes 7 was a good performance 
indicator; in contrast, for the last class the values of 7 misleadingly suggest that 
both group structures perform equally well. Additionally, for a set of image 
processing experiments we performed but omit due to space constraints, 7 was 
not a useful indicator of performance for group structures of practical interest. 
This is indicative of the potential looseness of the bound provided by Theo- 
rem [2j We believe that such looseness is a characteristic of guarantees that 
rely on worst-case metrics, such as the coherence n(A) and our grouped metric 
7(A, T, G), and is compounded by the looseness in the estimate of j(A,T,G) 
obtained via Theorem [3] (of up to 21%). 

5 Conclusions and Future Work 

In this work, we have presented an analytically derived multiplicative penalty 
on the number of measurements needed for CS recovery when the measurements 
exhibit grouped structure instead of the usual independently drawn measure- 
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Figure 2: 7 vs. M/Mq for group structures Q 1 and Q 2 when Fourier coefficients of 5% 
sparse signal s were concentrated within: (top) a sub-band built out of two 5%-wide 
channels; (middle) a sub-band built out of four 5%-wide channels; and (bottom) the 
entire band. 



ment assumption taken by most existing CS literature. Such grouped sampling 
is of large practical interest as full randomization of measurements is difficult to 
achieve in many compressive sensing acquisition systems. A notable limitation 
of the introduced penalty factor 7 is that it is dependent on the signal support. 
We expect further work to focus on penalty metrics that are independent of 
the support of the signal being measured, and to expand the guarantees pro- 
vided to more applicable approximately sparse signals and to noisy measurement 
schemes. 



A Proof of Theorem [2] 

In this appendix, we will prove the result of Theorem [2] with a small modifica- 
tion on the distribution of the submatrices: Instead of a uniform distribution 
among all subsets f2 containing M/G out of the N/G available groups, we pose 
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an independent Bernoulli selection for each group submatrix Gi, i = 1, • • • , N/G, 
belonging in O with selection probability 6i = M/N. This independent model 
results in the expected number of selected groups being equal to M/G. Fur- 
thermore, one can show that since the probability of failure is a non-increasing 
function of the size M of the set fi, the probability of failure under the uni- 
form distribution used in Theorem [2] is upper-bounded by a constant times the 
probability of failure under the independent selection model used in the proof 
(a property dubbed poissonization in [7]). Thus, the effect of the conversion of 
the subgroup selection model is a constant multiplicative factor in the required 
number of measurements, which is accounted for by the constant in 

Following the argument of [T] , one can show that the signal c is the unique 
solution to Q if and only if there exists a dual vector tt £ H N that has following 
properties: 

• tt is in the row space of Aq , 

• 7r(t) = sign{c (i)} for t € T, 

• \n(t)\ < 1 for t € T c . 

As in pP, we consider the candidate 



where z is a |T|-dimensional vector whose entries are the signs of c on T. To 
prove Theorem [2] we need to show that under its hypothesis: (i) A^ t Aqt is 
invertible and (ii) \ir(t)\ < 1 for t £ T c . We begin by showing that Aq T Aqt is 
invertible with high probability given the requirement Q on M. In Theorem |4j 
we claim that if M is large enough then, on average, the matrix A^ t Aqt does 
not deviate much from ^J, where / is the identity matrix. 

Theorem 4. Fix an arbitrary subset T of the signal domain. Define N/G index 
groups Q = {Gi, ■ ■ ■ , Gn/g} of the measurement domain, each of size G, and 
draw each group independently at random into a set fl with probability M/N. If 



TT = A^AnT{Ai T A nT ) 1 z , 



(6) 





with ry(A,T,G) introduced in Q), then 





where \\-\\ denotes the spectral norm. 



Proof. Denote 
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where Si is a Bernoulli random variable with P(Si = 1) = Because A H A = I, 
we have 

N/G 

£ Ag T A GiT = I, (10) 

i=l 

and so we can write 



Y 



N 
M 



N/G 



M 
iV 



N/G 

^ViT^Q^T —■ E Yj. 



(11) 



i=l v ' ' i=l 

We will use Theorem 1.4 from [8], which we include below for completeness. 

Theorem 5. Consider a finite sequence {Y{\ of independent self-adjoint random 
matrices with dimension d. Assume that each matrix Yi satisfies E{Y"i} = and 
\\Yi\\ < B almost surely. Then, for all t > 0, 



P- 



For our case, 



> t > < d ■ exp 



Yi 



V2_\ 



a 2 + Bt/3 J 



where a 



M 
iV 



N 
M 



(12) 



and E(Yi) = 0. We find a bound B on 



\YA\ = 



sup |(/i,yj 2 )l < sup 

/l./2 /l>/2 



/i,T7 E° ®a l f 2 



M 



leQi 



a ; (g) a' 



/a)l = ^f u PEK/i' 0, )< o, ,/2> 



A, /a 



< 



N 
M 



12 leGi 11 11 



(13) 



where the supremum is over unit-norm vectors f\ and f 2 , a 1 is the I th row of 
the matrix At, and 7 is defined in ([3]). 

Next, we calculate a 2 from Theorem [5] as 



N/G 

E E ( y * 2 ) 



= var{5;} 



M 2 



N/G 

j2(Ag T Ag iT f 



(14) 



Since Ag. T Ag.x is a Hermitian matrix, its eigendecomposition is Ag, T Ag.T — 
ftiAiilf , where is a matrix whose columns arc the orthonormal eigenvec- 
tors ujij, j — 1, . . . , \T\, of the matrix Ag.rpAg^, and Aj is a diagonal matrix 



containing the eigenvalues {AyJ-j-Jj of the matrix Ag^Ag.rp. Thus, we can write 

_ r> *2nH _ \2 , , , .H 



>3- 



E a 

J=1,..,|T| 



ijUijUlj' (I 5 ) 
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and so 



N/G 



i=X 



N/G 

E E Kj^'.^'j 

i=l j=l,...,|T| 



2 0J;jUJ H 



(16) 



The right side of ( 16 ) is a weighted double sum of positive semidefinitc matrices. 
The spectral norm of such a sum increases monotonically with the value of each 
of the weighting coefficients. Therefore, we can upper-bound ( 16 ) by replacing 
Xij with max i=1 n max J=1 ,...,\t\ ^ij an d taking it out of the operator norm: 



N/G 



< max max 

i=l,...,« j=l,...,\T\ ■ 



N/G 

E E a, 

i=l j=l,...,|T| 



,H 



= max || Ag. T Ag.T\\ ■ 



N/G 

E A G,T A G,T 



With (17) we can bound (14) by 



N 2 



a 2 < var{<5J — max \\Ag T Ag. T \\ ■ 



N 



N/G 

E A g t T A QiT 



vax ^>M? W A SiT A sM\ ■ \\A?A T \ 



M 
~N 

N 



M\ N 2 ... ^ , , 

~N J M 2 i-?^" ii f M S1 j!? H , i(/l 'L a8a ^ 

/ 4 — 1,.... G |Ul || = || /2 11=1 



- a7 _ max « sup E l^ 1 ' '® ^ 



< 



< 



M ^-S\\h\\=\\h\\^i e g i 

^- max sup V \{fi,a l )\ ■ \{a\h 
M i=i,...,f liAiHIAIHijtft 

-% 2 W| max^ sup £ 



M 

iV 
M 



■% ll/i||=i 



M 2 (A)|T| 7 = J B. 



We put together ([B]), Q and Theorem [5] to write 



(17) 



(18) 



P(||y||>l/2)<|T|-exp 
which proves Theorem [4] 



7/6-f Ai 2 (A)|T| 7 



(19) 
□ 



11 



Theorem [4] shows that if M is large enough, then Aq T Aqt is invertible with 
high probability. We continue by proving that |7r(t)| < 1 for t € T c . Following 
the techniques in [1], we need to prove four lemmas: 

Lemma 1. Denote by v° a row of the matrix A^A^x indexed by t 6 T . 
Then 

£|n| 2 <-^ 3 (A)|T| 7 . (20) 
Proof. One can express v° as 



N/G 

v " = E^E A (^°V- 

Now due to the orthogonality of the columns of the matrix A, 



(21) 



N/G 

^^A(Mo)A(M) = o, 

i=i zee, 



(22) 



and we can write 



N/G N/G 

i=i ieSi i=i 



(23) 



with 



Y, 



M 

iV 



E^cmoV 



We see that -E(i'i) = and we can write 

2 



= E 



N/G 



/N/G N/G \ 

WE y >E y 

\ i=i i'=i / 



W/G JV/G iV/G 

/•. E > • /- E < y ' y, > = E £ ( y *> y > • 

2—1 i^i' i—1 



Each element of the sum above can be bounded by 



(24) 



(25) 



E(Xi,Yi) = E((6 i - I ^)^2A(l ) t )a l A6^ 



N 



M 



E to)* 1 ' 



var^W Y>Ml,toW, E A(Z',t ) 
M 



a 



M 
iV 



1 



N 



E^»*o) E ^M(aV 
fee; i'eSi 



12 



< 



M 
iV 



M 



a',/ 



-'a 1 ' 



J a 1 



iV 



< 



< 



,M 



(^)m-Ei^,*o)i E 



J a' 



AT 



M 3 (A)|T|- 7 EKMo)|. 



(26) 



Putting together ( 25 1 and ( 26 ) , we get 



M 



N/G 



E\\v^ < ^ M 3 (A)|T| 7 EEl^*o)l 



i=i re Si 

A? 



■V(^)m7E|Aa,to)| = ^ 8 (A)|T| 7 ||A(:,to)|| 1 . (27) 



Now since A H A = /, we have ||A(:,io)|| 2 — 1) an d since for any vector h of 
length N we have \\h\\i < -/N\\h\\ 2 , it follows that 

M 



which proves Lemma IT] 
Lemma 2. Define 



a := 7/i (^)^- • max 



£||z,°|| 2 <-= M 3 (A)|T| 7 , 



M 



{i,V7m 3/2 (^ 3/4 |t|/Vm}. 



For < a < 



<l a nrf0< fl <f 

VM - V 



M 



P(\\A\ > ^ /2 {A)N- 1 ' i ^M\f\ +aaj < 3e~ 

for some positive constant k. 
Proof. By definition, 



(28) 
□ 

(29) 

1/4 

if 
(30) 



N/G 



\\v°\\= sup <t,°,/) = sup E( r -/)' 



(31) 



i=l 
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with v° from ( |23| ) and Yi from (24 1. For completeness, we reproduce below 
Theorem 3.2 from pQ, which we use to prove Lemma [2j 

Theorem 6. Let Yj, . . . , Yjy &e a sequence of independent random variables tak- 
ing values in a Banach space and let Z be the supremum Z = supy e jr Yl n =i f 0^i)> 
where T is a countable family of real-valued functions. Assume that |/(V)| < B 
for every f G T and all Y , and E/(Yi) = for every f € T and i = 1, . . . , JV. 
Then, for all t > 0, 



P(\Z-EZ\ > t) < 3exp 



log 1 



(32) 



merical constant. 



a 2 + BEZ / 

, and K is a nu- 



Denote the mapping (Yi, f) for a fixed unit vector / as /(Yi), so that Z 
suPn/ii^Etf/m) = We have E{/(Y,)} - 0, and 



\M)\ 



Si - 



M 
iV 



Y,A{l,t )a l J 



leQi 



Si- 



M 
iV 



leGi 



< 



Y,A(l,t )(a l ,f) 



leg, 



< max\A(l,t )\ ■ max||a'|| • V 



\(of,f)\ 



< 1 -n 2 (A)-^\T\=:B. 
Now we find a bound on E{/ 2 (Yj)}: 



(33) 



E{/ 2 «)} 



E 



var{(5,} 



71/ 

iV 

71/ 



M 

TV 



Y,A(l,t )a l ,f 



leg, 



J2Ml,to)(a l ,f) 

leSi 



M 
M 



Y,A{l,t )(a l ,f) 



teg. 



< ^ 2 (^)£i<^/>r, 



leg. 



and so 



N/G 



^ E {/ 2 (r,)}<^)|EEK tt '/) 



N/G 



N 



i=i /e5i 



We know that Digc- I ( a '' /) I 2 = th ere f° re i 



N/G 



/V' 



(34) 
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Plugging ( 33 1 and ( 34 ) in Lemma [TJ we have 



E{Z} = E{\\v°\\} < M 3/2 (A)v ^|^ v 



NV*' 



(35) 



Assume that v^lTV 3 ^)^ 3 / 4 < ]_ and < a < 



we have 



^^A)s/N^\T\ 



then with (29) 



a 2 = 7M 2 (^)f > BE{Z} = BE{\\v°\\} > B^'\A)^^\j^_, 



and by writing t = ao~ we have 



Bt = Baa < a 2 . 



(36) 



(37) 



For 



77|T|M 3/2 (A)jV 3 / 4 



'M 



> 1 and < 



a - {j^a)Vn) 



1/4 



, with ( 29 ) we have 



-2 = ^nV^A)\T\y/M = BE{Z} = BE{\\v°\\} 



A I 



iVV4 



and so 



Bt < a 2 . 

Putting together @, @, and Theorem [6| we can write 

(||t; || > A/ 3 / 2 (A)7V- 1 / 4 v ^iWM + a ( r) < 3 e - KQ2 



P 



(38) 
(39) 

(40) 



where ;> i> a numerical constant /.■ = log ^' 5 ) and K comes from Theorem [6j 



This completes the proof of Lemma [2j 



□ 



Lemma 3. Let w° — (Aq T Aqt) With the notations and assumptions of 

Lemma\^we have: 



\T\ 2Naa\ 



< ne -W +P (\\A* T A nT \\< 



M 
2N 



Proof. Denote the events 



\Aq,tAq.t\\ > 



M 
2N 



and 



E 2 : \ sup \\v°\\< f i 3/2 (A)N- 1 / i ^MfT\+, 
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We can write 

P ( sup ||w°|| > 2N 3/4 n 3/2 Vl\ T \/ M + 2Naa/M) < P(E 1 n E 2 ) 

= P(E 1 UE 2 ) 



(41) 



and with Lemma [2] we have 



P sup \\w°\\>2N 3 /Y / \ l 

\toGT= V M 



7|T| 2iVao-\ 



< P ||^ t Aot|| < 



M 
27V 



3e" 



(42) 
□ 



Finally we will use Lemma 3.4 from [T] 



Lemma 4. Assume that z(t), t £ T is an i.i.d. sequence of symmetric Bernoulli 
random variables. For each A > 0, we have 



P ( sup |7r(t)| > 1 ] < 2ne~ 1/2x2 + P ( sup ||u; | 



> A 



(43) 



Now that all lemmas are in place, we are ready to prove Theorem [2j If we 
pick A = 27V 3 / 4 /i 3/ V7l T l/ M + 2Naa/M in Q, from ^ and Q we get 

P (sup |7r(i)| > 1] < 2Ne~ 1/2x2 +Ne- Ka2 +P (\\Ag T A nT \\ < M/2N)) . (44) 

For the right hand side of ( 44 ) to be smaller than 3<5 we need all three summands 
to be smaller than 8. We now derive conditions on S that provide this guarantee. 
We start with the second summand: for it to be no bigger than S we can set a 2 
to be 



a 2 = K-Hog(N/6). 
For the first summand to be no bigger than <5, we need 

1 



A 2 



>2log(2N/S). 



(45) 



(46) 



If 



V7^ 3/2 (^)A f3/4 |T| 



> 1, Lemma 2 requires 



< a < 



( M 



\j^(A)VN 



1/4 



(47) 



Then with a from ( 29 1 we get 



Naa/M < /i 3 / 2 AT 3 /V 7 |T|/M, 



(48) 
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and so 

A < ^ 3 ^{A)N 3 / 4 y/^fr\/M. 
Reorganizing terms, we obtain 



1 



> 



M 



A 2 " 16Ai 3 (A)7V 3 / 2 7 |T| 



(49) 
(50) 

(51) 

Suppose now that Vw 3/2 (Ayv 3/4 m < \ Therij with (29l, ct 2 = 7^ 2 (A)^. If 



From (45) and (47 1 we get the following bound on M: 
M > 7 n(A) VNk~ 2 log 2 (AT 5). 



fi(A)y/N\T\ > a 2 , then 
and 

and thus 



Naa/M < fi 3/2 N~ 1/4 y^\f\/M, 
A < ^ 3 ' 2 (A)N 3 / 4 ^ 1 \T\/M, 



> 



M 



A 2 " 16fi 3 (A)N 3 / 2 7\T\' 



(52) 
(53) 
(54) 



which matches the previous condition (50|. On the other hand, if ^j,(A)\' N\T\ < 



then 



and 



Naa/M > (i 3/2 N~ 1/4 y/~/\T\/M, 
A < ANaa / M, 



and thus, with a 2 = ~/[A 2 (A)jjr, 



1 



> 



M 2 



M 



A 2 " 16A^ 2 a 2 CT 2 Wa 2 ~ffi 2 {A)N' 



And so with ( 54 ) and ( 57 ) we can write 
M 



1 1 

/x(A)An/2|r|' a 2 



>21og(2JV/<J), 



167/i 2 (A)A/ 

M> 16^/j/ 2 (A)Nmax(jj,(A)N 1/2 \T\,a 2 ) 21og(2AT/<5), 



which with ( 45 1 gives 

M > Const • J(J?(A)N max ( fi(A)N 1/2 \T\, log 



log 



Due to Theorem [4j for the third summand to be smaller than S, we need 



M>|.7-W^ 2 ^)-|r|log^ 



Thus from (51 1, (60 1 and (61 ) we see that the overall requirement on M is: 

M > Const • j(A,T,Q) ■ fi 3 {A)N 3 / 2 \T\\og(N/S), 
which finishes the proof of Theorem [2] 



(55) 
(56) 

(57) 

(58) 
(59) 

(60) 

(61) 
(62) 
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